Application No. 10/716,616 Docket No.: 0142-0436P 

Amendment dated October 15, 2008 
After Final Office Action of June 16, 2008 

AMENDMENTS TO THE CLAIMS 

1. (Currently Amended) A method of segmenting a composite image of pixels into a 
number of fields corresponding to lay-out elements of the image, the pixels having a value 
representing the intensity and/or color of a picture element, the method comprising: 

anal yzing the pixels of the image as either background pixels having a background 
property indicative of a background of the image or foreground pixels not having said 
back ground property; 

detecting in the image any foreground separator elements that are objects in the 
foreground of the image having a pattern of foreground pixels; 

replacing the pixels of the detected foreground separators by pixels having the 
background property; 

constructing separating elements corresponding to rectangular areas of adjacent pixels of 
the image having [[a]] the background propert y indicative of a background of the image ; and 
constructing a graph representing the lay-out elements of the image by 

- defining vertices of the graph on the basis of intersections of separating elements 
that are substantially oriented in predetermined separation directions, 

- defining edges of the graph between the vertices corresponding to the separating 
elements, and 

- defining field separators corresponding to the edges of the graph. 

2. (Original) The method as claimed in claim 1, wherein the step of defining vertices 
comprises: 
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constructing subsets of the separating elements that are substantially oriented in the 
predetermined separation directions, and 

determining the intersections between pairs of separating elements from the subsets. 

3. (Original) The method as claimed in claim 2, wherein the step of determining the 
intersections comprises: 

detennining an area of overlap of the separating elements from the subsets, and 
locating the vertex at the center of the area of overlap. 

4. (Original) The method as claimed in claim 1, wherein the graph constructing step 
comprises: 

assigning a weight to the edges indicating an Euclidean distance between the vertices. 

5. (Original) The method as claimed in claim 1, further comprising: 

constructing a set of maximal rectangles, a maximal rectangle being a rectangular part of 
the image in one of the separation directions, that has the maximum possible area without 
including a pixel not having the background property indicative of a background of the image; 
and 

constructing the separating elements in a cleaning step wherein at least one pair of 
overlapping maximal rectangles in the set is replaced by an informative rectangle that is a 
rectangular part of an area combining the areas of the pair, the rectangular part having the 
maximum possible length in the relevant separation direction. 
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6. (Original) The method as claimed in claim 5, wherein the cleaning step further 
comprises at least one of the following: 

deleting a maximal rectangle having a length below a predefined value, 
deleting a maximal rectangle having a width below a predefined value, and 
deleting a maximal rectangle having an aspect ratio below a predefined value, the aspect 
ratio being a longer side length divided by a shorter side length. 

7. (Original) The method as claimed in claim 5, wherein, prior to the step of constructing 
the maximal rectangles, the image is preprocessed by at least one of the following: 

removing noise by adapting the value of isolated deviant pixels to an average value of 
pixels in the neighborhood, 

halftoning by transforming the pixels to either white or black, and 
reducing the number of pixels by downsampling. 

8. (Cancelled) 

9. (Currently Amended) The method as claimed in el aim 8 claim 1 , wherein the 
foreground separator elements include black lines, dashed lines, or dotted lines. 

10. (Original) The method as claimed in claim 5, wherein the step of constructing the 
maximal rectangles comprises; 
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determining a list of maximal runs, a maximal run being a straight line of pixels having 
the background property, the line having the maximum possible length without including a pixel 
not having the background property, 

taking a specific maximal run from the list as rectangle, 

testing the rectangle if extension is possible by determining for a next maximal run if the 
next maximal run comprises pixels adjacent to pixels of the rectangle in a width direction, 

if the extension is possible, extending the rectangle by constructing a new rectangle 
having the maximum area including pixels of the rectangle and the next maximal run, 

if no extension is possible, adding the rectangle to the set of maximal rectangles, and 

eliminating from the list any maximal run that is completely contained in the new 
rectangle. 

11. (Original) The method as claimed in claim 1, wherein the step of constructing the 
separating elements comprises: 

processing the image in two orthogonal separation directions. 

12. (Original) The method as claimed in claim 1, wherein the step of constructing the 
separating elements comprises: 

detecting graphical elements that are objects in the foreground of the image having a 
pattern of pixel values deviating from said background property, wherein the separating elements 
are constructed around the graphical elements. 
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13. (Original) The method as claimed in claim 1, wherein at least one of the fields is 
classified as text field, a reading order is detected in the text field, and foreground components 
are joined to text lines in the text field in a direction corresponding to said reading order. 

14. (Currently Amended) A computer program product embodied on at least one 
computer readable medium, for segmenting an image of pixels into a number of fields, 
corresponding to lay-out elements of the image, the pixels having a value representing the 
intensity and/or color of a picture element, the computer program product comprising computer- 
executable instructions for: 

analyzing the pixels of the image as either background pixels having a background 
property indicative of a background of the image or foreground pixels not having said 
background property; 

detecting in the image any foreground separator elements that are objects in the 
foreground of the image having a pattern of foreground pixels; 

replacing the pixels of the detected foreground separator elements by pixels having the 
background property; 

constructing separating elements corresponding to rectangular areas of adjacent pixels of 
the image having [[a]] _the background propert y indicative of a - background of the image ; and 
constructing a graph representing the lay-out elements of the image by 

- defining vertices of the graph on the basis of intersections of separating elements 
that are substantially oriented in predetermined separation directions, 
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- defining edges of the graph between the vertices corresponding to the separating 
elements, and 

- defining field separators corresponding to the edges of the graph. 

15. (Original) The computer program product as claimed in claim 14, wherein the 
computer-executable instructions for defining vertices comprise computer-executable 
instructions for: 

constructing subsets of the separating elements that are substantially oriented in the 
predetermined separation directions, and 

determining the intersections between pairs of separating elements from the subsets. 

1 6. (Original) The computer program product as claimed in claim 14, wherein at least one 
of the fields is classified as text field, a reading order is detected in the text field, and foreground 
components are joined to text lines in the text field in a direction corresponding to said reading 
order. 

17. (Currently Amended) The computer program product as claimed in claim 14, further 
comprising computer-executable instructions for: 

constructing a set of maximal rectangles, a maximal rectangle being a rectangular part of 
the image in one of the separation directions, that has the maximum possible area without 
including a pixel not having the background property indicative of a background of the image; 
and 
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constructing the separating elements in a cleaning step wherein at least one pair of 
overlapping maximal rectangles in the set is replaced by an informative rectangle that is a 
rectangular part of an area combining the areas of the pair, the rectangular part having the 
maximum possible length in the relevant separation direction. 

18. (Currently Amended) A device for segmenting a composite image of pixels into a 
number of fields corresponding to lay-out elements of the image, the pixels having a value 
representing the intensity and/or color of a picture element, the device comprising: 

an input unit for inputting an image; and 

a processing unit fo r analyzing the pixels of the image as either background pixels having 
a background property indicative of a background of the image or foreground pixels not havin g 
said background property; detecting in the image any foreground separator elements that are 
objects in the foreground of the image having a pattern of foreground pixels; replacing the pixels 
of the detected foreground separators by pixels having the background property; and constructing 
a graph representing the lay-out elements of the image by 

- constructing separating elements corresponding to rectangular areas of adjacent 
pixels of the image having [[a]] the background propert y indicative of a background of 
the image , 

- defining vertices of the graph based on intersections of separating elements that 
are substantially oriented in different separation directions, and 

- defining edges of the graph between the vertices corresponding to the separating 
elements. 
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19. (Original) The device as claimed in claim 18, further comprising: 
a display unit for displaying fields of the image after segmenting. 

20. (Original) The device as claimed in claim 18, wherein the processing unit constructs a 
set of maximal rectangles, a maximal rectangle being a rectangular part of the image in one of 
the separation directions, that has the maximum possible area without including a pixel not 
having the background property indicative of a background of the image; and constructs the 
separating elements in a cleaning step wherein at least one pair of overlapping maximal 
rectangles in the set is replaced by an informative rectangle that is a rectangular part of an area 
combining the areas of the pair, the rectangular part having the maximum possible length in the 
relevant separation direction. 

2L (Original) The device as claimed in claim 18, wherein at least one of the fields is 
classified as text field, a reading order is detected in the text field, and foreground components 
are joined to text lines in the text field in a direction corresponding to said reading order. 

22. (Previously Presented) The method as claimed in claim 1, wherein the rectangular 
areas are white areas. 



23. (Currently Amended) The computer program product as claimed in claim 14, wherein 
the rectangular areas are white areas. 
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24. (Previously Presented) The device as claimed in claim 18, wherein the rectangular 
areas are white areas. 
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